A First-Order Approach to Accelerated Value Iteration
نویسندگان
چکیده
Markov decision processes (MDPs) are used to model stochastic systems in many applications, but computing good policies becomes hard when the effective horizon become very large. In “A First-Order Approach Accelerated Value Iteration,” Goyal and Grand-Clément present a connection between value iteration (VI) algorithms gradient descent methods from convex optimization use acceleration momentum design faster algorithms, with convergence guarantees for computation of function fixed policy reversible MDP instances. The authors provide lower bound on properties any first-order algorithm solving MDPs, where no can converge than VI. Finally, introduce safe accelerated (S-AVI), which alternates updates updates. S-AVI is worst-case optimal retains theoretical VI while exhibiting strong empirical performances providing significant speedups compared classical approaches large test bed
منابع مشابه
Accelerated Extra-Gradient Descent: A Novel Accelerated First-Order Method
We provide a novel accelerated first-order method that achieves the asymptotically optimal con-vergence rate for smooth functions in the first-order oracle model. To this day, Nesterov’s AcceleratedGradient Descent (agd) and variations thereof were the only methods achieving acceleration in thisstandard blackbox model. In contrast, our algorithm is significantly different from a...
متن کاملThe variational iteration method for a class of tenth-order boundary value differential equations
متن کامل
A numerical approach to solve eighth order boundary value problems by Haar wavelet collocation method
In this paper a robust and accurate algorithm based on Haar wavelet collocation method (HWCM) is proposed for solving eighth order boundary value problems. We used the Haar direct method for calculating multiple integrals of Haar functions. To illustrate the efficiency and accuracy of the concerned method, few examples are considered which arise in the mathematical modeling of fluid dynamics an...
متن کاملA Lyapunov approach to Accelerated First Order Optimization In Continuous and Discrete Time
A Lyapunov Approach to Accelerated First-Order Optimization In Continuous and Discrete Time
متن کاملAccelerated First-order Methods for Hyperbolic Programming
A framework is developed for applying accelerated methods to general hyperbolic programming, including linear, second-order cone, and semidefinite programming as special cases. The approach replaces a hyperbolic program with a convex optimization problem whose smooth objective function is explicit, and for which the only constraints are linear equations (one more linear equation than for the or...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Operations Research
سال: 2023
ISSN: ['1526-5463', '0030-364X']
DOI: https://doi.org/10.1287/opre.2022.2269